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ANALYSIS  APTITUDK  TKST  FOR  SELECTION  OF  AIRMEN  FOR  THE  RADIO 
COMMUNICATIONS  ANALYSIS  SPECIALIST  COURSE: 
DEVELOPMENT  AND  VALIDATION 


I.  INTRODUCTION 

The  Radio  Communications  Analysis  Specialist 
course  administered  by  the  USAF  School  of 
Applied  Cryptologic  Sciences  at  Goodfellow  Air 
Force  Base,  Texas,  is  the  basic  course  for  several 
Air  Force  and  Amiy  personnel  specialties.  These 
intelligence  occupations  require  personnel  of  very 
high  mental  ability,  and  experience  relatively  high 
training  attrition  rates  (about  20^  in  the  20210 
course).  Selection  of  Air  Force  enlistees  for  this 
career  field  currently  includes  a minimum  General 
Aptitude  Index  (Gen  Al)  percentile  of  80.  The 
Gen  Al  is  composed  of  the  Word  Knowledge  (WK) 
and  Arithmetic  Reasoning  (AR)  subtests  of  the 
Armed  Services  Vocational  Aptitude  Battery 
(ASVAB). 

The  duties  of  Radio  Communications  Analysis 
Specialists  require  considerable  analytical,  reason- 
ing, and  verbal  skills.  This  is  apparent  from  the 
following  excerpts  from  the  Duties  and  Responsi- 
bilities section  of  AFR  39-1,  AFSC  20230,  dated 
June  1977; 

Compiles,  records,  and  analyzes  radio  fre- 
quency data.  Prepares  and,  studies  circuit 
diagrams,  call  signs,  and  operating  character- 
istics. . . . Reconstructs  radio  networks  and 
draws  net  diagrams  in  schematic  or  geo- 
graphical design.  Determines  radio  network 
schedules,  personalities,  and  procedures 
characteristics.  Compares  and  categorizes 
messages  through  analysis  of  message 
internals,  externals,  and  other  textual 
features. 

Personnel  at  Goodfellow  AFB  have  developed 
an  Analysis  Aptitude  test  for  the  purpose  of 
supplementing  the  Gen  Al  in  selection  of  Air 
Force  20210  students.  The  objective  of  this  study 
is  to  assess  the  psychometric  characteristics 
(especially  reliability  and  validity)  of  the  Analysis 
Aptitude  (AA)  test  and  its  relationship  to  the  Gen 
Al. 


II.  METHOD 

Subjects 

Three  samples  were  utilized  in  the  study. 
Sample  1 included  301  Air  Force  enlistees  from 
randomly  selected  flights  who  were  tested  during 


basic  training.  Data  from  this  sample  were  used  to 
measure  the  general  level  of  performance  on  the 
AA,  AA  item  difficulty  levels,  and  AA  lest  reli- 
ability. Sample  2 consisted  of  173  ainnen 
attending  the  20210  course  and  for  whom  validity 
data  were  obtained.  Sample  3 was  comprised  of 
144  Army  20210  students.  This  sample  was  used 
to  compare  AA  scores  with  the  Gen  Al.  Since  the 
Air  Force  selects  20210  students  using  a Gen  Al 
80  percentile  criterion,  these  scores  are  extremely 
attenuated.  Corrections  for  restriction  in  range 
would  be  inaccurate  unless  very  large  samples  were 
used.  Also,  tlie  Gen  Al  is  normally  recorded  in 
5-point  percentile  intervals.  Thus,  only  four 
rectangularly  distributed  scores  (i.e.,  80,  85,  90, 
95)  are  in  the  range  for  ainnen  qualified  for  the 
20210  course.  Anny  enlistees,  however,  were 
selected  on  the  basis  of  a skilled/technical 
composite  which  contains  the  Mathematics  Knowl- 
edge, Arithmetic  Reasoning,  and  (ieneral  Science- 
Biology  ASVAB  subtests.  Therefore,  Gen  Al 
scores  of  Army  20210  students  will  be  less 
attenuated  than  those  for  Air  Force  students. 

Predictor  Variables 

Some  characteristics  of  the  Analysis  Aptitude 
test  are  shown  in  Table  1 . Two  sample  items  arc 
given  in  Table  Al  of  the  appendix.  ASVAB 
subtesis  WK,  AR,  and  Space  Perception  (SP)  also 
were  predictors  in  the  study.  The  Air  Force  Gen 
Al  is  composed  ofWK  and  AR.  The  Armed  Forces 
Qualification  Test  (AFQT),  used  to  determine 
eligibility  for  Air  Force  entry,  consists  of  WK.  AR. 
and  SP.  The  current  ASVAB  forms  arc  described 
elsewhere  (Jensen,  Massey, & Valentine,  1976). 


Table  I.  Analysis  Aptitude  Test  Characteristics 

(liatpd  on  Kamlom  Sample  of  301  Air  horre  I ntuleet) 


Number  of  itern^ 

22  four-choice  items 

Testing  lime 

45  minutes  ♦ 10  minutes 

for  directions 

Average  item  difl'icully  Ip) 

.65 

p range 

.40  to  85 

Reliability  (KR-JO) 

.78 

Mean 

I4..t 

Standard  Deviation 

4..1 

5 

( 
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Training  Criteria 

Final  grade  in  the  20210  course  was  the 
primary  criterion.  Pass/fail  (P/F)  status  was  also 
used,  but  the  variance  of  this  variable  was  low,  due 
to  the  majority  of  students  being  in  the  pass 
group.  In  the  Air  Force  sample,  81. 5'7  passed,  and 
in  the  Anny  sample  84.0'^  passed.  For  statistical 
purposes,  the  I'inal  grade  criterion  will  be  a more 
accurate  indication  of  predictor  validity  for  the 
20210  course. 

Statistical  Method 

All  predictor  and  criterion  information  were 
obtained  from  the  USAF  School  of  Applied 
Cryptologic  Sciences  with  the  exceptions  of  those 
pertaining  to  the  sample  on  301  Air  Force  en- 
listees and  the  ASVAB  subtest  data.  ASVAB 
scores  came  from  the  U.S.  Army  Recruiting 
Command  file  (USAREC-1),  a copy  of  which  is 
maintained  by  the  Computational  Sciences 
Division  of  the  Air  Force  Human  Resources 
Laboratory. 

Analyses  included  simple  and  multiple  concla- 
tions.  Corrections  for  restriction  in  range  were 
made  when  appropriate  via  formulae  (Guilford  & 
Fruchter,  1973).  Tlie  F statistic  was  employed  to 
test  tlie  significance  of  the  increase  in  a multiple 

when  additional  predictors  were  used. 


III.  RHSl  LTS  AND  DISCI  SSION 

Based  on  administration  to  Air  Force  enlistees, 
the  Analysis  Aptitude  (AA)  test  had  a mean  of 
14.3  and  a standard  deviation  (SD)  of  4.3.  Means 
and  SDs  of  study  variables  are  listed  in  Table  A2 
of  the  appendix.  Tlie  reliability  of  the  AA  was  .78 
as  computed  by  the  Kuder-Ricbatdson  20  formula 
(Table  1).  Levels  of  AA  performance  of  the 
enlistees  are  shown  in  Table  2.  Tire  most  frequent 
score  was  14.  and  scores  above  14  were  achieved 
by  48.8' J . 

Tlie  validities  of  the  AA  and  (ien  Al  for 
samples  are  presented  in  Tabic  3.  High  validities 
were  obtained  for  AA  with  20210  final  grades,  .58 
for  Air  Force  and  .54  for  Army  students.  Lxiwer 
validities  were  obtained  for  Gen  Al,  .25  for  Air 
Force  and  .40  for  Army  students.  While  all  of 
these  correlations  (r's)  arc  underestimates  of  true 


Tahk  2.  Distributions  of  Analysis  Aptitude 
Scores  for  Air  Force  Enlistees 


A A Score 

Frequency 

% 

Cumul  Freq 

Cumul  % 

>_2 

K 

2.7 

301 

21 

16 

5.3 

293 

97.3 

2(1 

10 

6.3 

277 

92.0 

It 

20 

6.6 

258 

85.7 

IK 

16 

5.3 

238 

79.1 

17 

23 

7.6 

222 

73.8 

16 

23 

7 6 

199 

66.1 

15 

22 

7.3 

176 

58.5 

14 

24 

8 3 

154 

51.2 

1.1 

23 

7.6 

130 

43.2 

12 

22 

7.3 

107 

35.5 

11 

19 

6.3 

85 

28.2 

10 

18 

6.0 

66 

21.9 

9 

16 

5.3 

48 

15.9 

K 

15 

5.0 

32 

10.6 

7 

9 

3.0 

17 

5.7 

6 

4 

1.3 

8 

2.7 

5 

2 

0.7 

4 

1.3 

4 

0.0 

2 

0.7 

,1 

2 

0.7 

2 

0.7 

Tahk  3.  Validities  (Uncorrected)  of  AA 
and  Gen  Al  for  20210  Course 

Validity  (7) 

Fi  nai 

Pass/ 

Sample 

N Grade 

Fall 

Analysis  Aptitude  Test 

Air  Force 

173  .58** 

.56** 

.Aniiy 

144  ..54“ 

.40** 

General  Aptitude  Test 

Air  Force 

55-'  .25 

.31* 

Army 

144  .40*  ♦ 

.33** 

‘'Subs.iinplc  witii  ASVAB  scores  available, 
•p  < .05. 

'•p<  .01. 


relationships  due  to  attenuation  effects,  the  Gen 
Al  is  more  severely  affected  because  of  its  use  in 
screening.  Tlie  validity  of  AA  with  P/F  was  .56  for 
Air  F-'orce  and  .40  for  Army  students.  The  r for 
Gen  Al  with  P/F  was  .31  for  Air  Force  and  .33  for 
Army  students. 

Correlations  of  AA  and  subtests  contained  in 
(ien  Al  with  final  grades  (intcrcorrclation  matrix  is 
in  Table  A3)  were  corrected  for  restriction  in 
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range  for  (he  Army  sample.  Subtest  data  were 
available  for  108  students.  After  correction,  AA 
validity  rose  somewhat,  from  .54  to  .59  (Table  4). 
Larger  increases  were  obtained  for  both  Gen  Al 
subtests  as  the  validity  of  WK  increased  from  .39 
to  .58  and  AR  validity  increased  from  .30  to  .44. 
The  multiple  correlation  (R)  obtained  when  con- 
tributions of  WK  and  AR  were  added  to  that  of 
AA  for  prediction  of  final  grades  was  .69.  This 
represents  a significant  increase  in  prediction 
(/)<.01)  over  that  obtained  with  AA  alone.  An 
additional  significant  increase  is  obtained  when  SP 
is  added  to  AA,  WK,  and  AR,  the  accounting 
for  an  additional  4%  of  the  variance.  Tltis  indicates 
that  the  AFQT  composite  (consisting  of  WK,  AR, 
and  SP)  could  be  substituted  for  Gen  Al  as  a co- 
selection instrument  along  with  AA. 


Tah/c  4.  Test  Validities  with  Final  Grades 
for  Army  20210  Students 

(.V  = lOH) 


Tests 

Validity* 

y tc 

Multiple  R 

Combi- 

nation 

F*> 

1 Analysis  .Aptitude 

..S4* 

.59* 

1 

.35 

58.1* 

2 Word  Knowledee 

..39* 

.58* 

1 2 

.46 

21.4* 

3 Arithmetic 

ReasE'iTiing 

.30* 

.44* 

1-3 

.47 

1.8 

4 Space  Perception 

.3«* 

.38* 

1-4 

.51 

11.6* 

~ correlation 

corrcctc 

■d  for 

restriction 

in  range. 

^Test  tor  increase  in  . 

‘p  < .01. 


Table  5.  Performance  of  Air  Force  20210 
Students  at  Each  AA  Score  Level 


AA  Score 

N 

% 

Pats 

Mean 

Grade 

Cumul. 

N 

Cumm. 

% 

22 

9 

lOO.I) 

90.3 

173 

100.0 

21 

20 

100.0 

88.6 

164 

94.8 

20 

14 

100.0 

86.3 

144 

83,2 

19 

16 

100.0 

86.1 

130 

75.1 

18 

19 

94.7 

85.5 

114 

65.9 

17 

1 1 

ItlO.O 

83.6 

95 

54,9 

16 

14 

92.9 

81.2 

84 

48.6 

15 

17 

88.2 

84.3 

70 

40,5 

14 

9 

55.5 

77.3 

53 

.30.6 

13 

8 

62.5 

77.6 

44 

25.4 

12 

15 

46.7 

73.7 

36 

20.8 

1 1 

4 

50.0 

69.3 

21 

12.1 

10 

6 

33.3 

63.2 

17 

9.8 

9 

5 

40.0 

73.4 

It 

6.4 

8 & Less 

6 

33,3 

71.0 

6 

3.5 

Total 

173 

81.5 

81.8 

not  have  met  these  requirements,  the  failure  rate 
was  39.5%.  If  the  AFQT  cutoff  was  80  and  the  AA 
cutoff  was  15,  then  the  failure  rate  would  have 
dropped  to  2.3%.  For  the  51  students  not  meeting 
these  criteria,  the  failure  rate  was  33.3%.  Since  all 
of  the  eight  additionally  screened-out  students 
passed  the  course,  an  AFQT  cutoff  percentfle  of 
71  appears  to  be  the  better  selector  score. 


IV.  CONCLUSIONS  AND 
RECOMMENDATIONS 


AA  test  frequency  distributions  and  20210 
course  pcrfomiance  averages  are  presented  for  Air 
Force  students  in  Table  5.  llie  most  efficient  A A 
cutoff  score  to  use  in  screening  appears  to  be  15. 
Ninety-seven  and  five-tenths  percent  of  those 
scoring  15  or  higher  passed,  and  88.2%  of  those 
with  a score  of  15  passed  the  course.  Only  55.5% 
of  those  obtaining  a score  of  14  passed,  and  just 
45. .3%  of  students  scoring  14  or  lower  passed.  Will) 
a cutoff  score  of  15.  30.6%  of  the  Army  20210 
students  would  not  have  qualified  for  this  course. 

The  probable  effect  of  employing  various 
combinations  of  cutoff  scores  on  the  AA  and 
AFQT  was  estimated  from  data  pertaining  to  the 
Army  sample.  If  an  AFQT  percentile  of  71  (there 
IS  no  score  of  70)  and  an  AA  score  of  1 5 were 
simultaneously  utili/.ed  as  minimal  selection 
criteria,  the  Army  20210  course  failure  rate  would 
have  been  2.9'!.  For  the  43  students  who  would 


The  psychometric  characteristics  of  the 
Analysis  Aptitude  test  appear  adequate  for  its  use 
in  selection.  Tlic  AA  scores  approximate  a normal 
distribution,  and  the  test's  reliability  (KR-20  = 
.78)  is  adequate  for  a 22-itcm  instrument.  Tire  AA 
test  demonstrated  high  validity  based  on  20210 
course  final  grades  (r  = .58)  and  pass/fail  (f=  .56) 
for  Air  Force  students.  The  AA,  Gen  Al,  and 
AFQT  tests  all  made  significant  unique  contribu- 
tions to  the  prediction  of  final  grades.  Based  on 
analyses  of  Army  20210  students  not  highly 
screened  on  cither  test,  it  is  recommended  that 
dual  cutoff  scores  of  an  AFQT  percentile  of  71 
and  an  AA  score  of  1 5 be  instituted  in  selection  of 
20210  students.  Since  the  AFQT  is  used  by  all 
services.  Army  20210  students  could  also  be 
selected  on  this  basis. 
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APPENDIX  A:  SUPPLEMENTAL  STATISTICS 


Tahk  AI.  Sample  Analysis  Aptitude  Questions 


Question  1 

Row  X 2 3 4 5 
RowY  4 7 10  13 

If  row  X were  continued  out  to  a value  of  9,  what  would  the  value  of  Y be  at  the  same  relative  position? 

A.  16 

B.  17 

C.  19 

D.  25 

There  are  six  post  offices  with  ZIP  codes  12188  thru  12193.  There  is  direct  mail  service  between  12188  and 
12189.  12193  and  12190,and  12192  and  12189, 12189  and  12191, and  12190  and  12189. 

Question  2 

Which  post  office  is  the  central  office  for  the  area? 

A.  12188 

B.  12189 

C.  12190 

D.  12191 


Table  A2.  Variable  Means  and  SDs  for  Various  Samples 


Simple 

N 

Analysis  Apt 

Gen  AI 

afqt 

Finil  Gride 

M 

SO 

M 

SO 

M 

SO 

M 

SO 

Air  Force  enlistees 

301 

14.3 

4.3 

39.2*’ 

__ 

71.4'* 

15.8 

Air  Force  20210 

173 

16.2 

4.0 

— 

— 

— 



81.8 

10.5 

Air  Force  202 1 0 

55“ 

16.0 

3.9 

42.6 

3.4 

72.9 

13.1 

82.6 

9.5 

Amiy  20210 

144 

15.9 

3.9 

42.1" 

— 

— 

— 

82.1 

10.8 

Anny  20210 

o 

00 

15.7 

4.0 

40.8 

6.9 

70.8 

17.4 

81.8 

10.6 

^Subsamplcs  with  ASVAB  scores  availabic. 

Converted  from  Air  Force  pcrcentUes. 
^'Converted  from  Army  Standird  Scores, 
*^PerccntiIe  Scores. 


J 
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Table  A3.  Intercorrelations  of  Predictors  and  Final 


Grades  for  Army  Sample 

= lOfi,  '>s  rorri*ff«*d /or  Rvstrirtvd  Rannvj 

— > 

Intercorrelations 

Variable 

AR 

SP  AA 

FG 

Word  Knowledge 

.76 

.56  .37 

.53 

Arithmetic  Reasoning 

.59  .46 

.44 

Space  Perception 

.09 

.38 

Analysis  Aptitude 

.59 

Final  Grade 

10 
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\l|{  lOKCI  III  M\N  KKSOniCKS  I,  MlOll A l l ilO 
llroiik^  \ir  l orci'  Ha~c.  IV\;i>  7K2.5') 

I'irralu 


Nuinlior 

KirsI 

Viillior 

Tillo 

\l'lll{|.-TH-7(i-«7  (Al»-A(l.{7  ri22) 

.li-uson 

Arinod  Sorvicos  Noialional  \|ililudo  liallon 
Ih’volofiiiioni  (.ASA  \U  Form-  ■>.  h.  and  7) 

\l  lll{|.-  l l{-77-2H  (AD-AOH  :)2:i) 

llunlor 

Validaliou  ol  a Ps\i'honiolor/I*ori'oplual  To-l  lialiori 

, M (n{|,-TI<-77-.');i  ( Al»-A(it«  12(1) 

Mai  how  . ■■ 

Soroonliif;  IV-I  liallori  lor  llonlal  l.alioralmv 
Spoi'ialisl  lioiirM’:  1 lo\ olo|nuonl  and  Validaliou 

‘ /f5-/?0‘r/y<l'3-'HlllL-TK-77-7MAI)-A():)l  ')(i2) 

1 

Malliows 

Vnaivsis  Vpliludo  I'o-I  lor  .'si  loc  lion  oi  Mriuon  lor 
llio  Radio  I ioin  munirai  ion-  \nalv-i-  Sponali-I 
lioui'M’:  DovoliipiuonI  and  Validaliou 

, M ll|{|,-'rH-7H-|()  (Al)-AOr>«  (»')7) 

J 

1 

DoVaiiv 

■Suppli  Rail- and  Fapiililiriuni  liui’nlors  ol  Vir  Fiirio 
Kniislod  Porsonnol:  V Siinullanoous  VIodol  id  llio 
Aooossion  and  Rolonlion  Vlarkol-  Iniorporaiini; 
F’oroo  Lovol  ('.oiistraints 

• \rill{|,-TH-78-7t  (A l)-AO(i(i  ()■>'») 

l-oiso\ 

Fliaraiiorislirs  ol  Vir  I'lirro  Viio-sion>:  Jaiiuan 
l*)75  lo  Juno  l‘)77 

\l  lll{|,-Tll-7«-82  (AD-AOli.l  oru,) 

Mai  hows 

l*rodii  lion  id  Roailiu^  Orado  l.ovol-ol  Sorvioo  Apjili- 
oanls  from  Ariuod  Sorviros  V oi  alioiial  Vpliludo  Ral- 
lory  (ASV  All) 

' M II|{|.-l'R-7')-2d  (Al)-A()78  127) 

llondrix 

Pro-FulisliuonI  I’orson-Joli  Vtali  li  S\-loni 

M mil  -■|■|^7')-8.^  (Al)-A(l'»(»  l<>')) 

Ouslalson 

llorur-iio  l•orol'a-lin};  Sisloni  lor  I’or-iin-Joh  Vlaloli 

Hiif  III  iioniiiii}!  |ir<ilili‘in>  l•ll^oullll•r^<l  with  ASV  All  Foriii>  A.  (i.  and  7.  piTriMilili'  '•^llrl■^  driiM'd  frnin 
ilii-M-  IrsI  liirm.-  an-  in  t‘rror.  VI  hllc  ihr  rrlaliv**  ranking  id  individuals  li\  llii'ir  |)rrriMilili-  srori's  winild  mil 
III’  alli’i’li’d  III  ihr  niirniin^  rrriir>.  llirir  ali'-iiliili'  sroci’  vallii’s  would  In’  diflrri’iil.  riii’irlori’.  di’M  rl|iliM' 
-lali'lir^  rr|iiirlrd  in  llir  siiliji’rl  li  rliuiral  rr|)iirls  abiivi’  arr  rrronroiis:  ollii’r  U [ii’'  iil  anali'i-s  in  llii- 
ri’|iiirl  wliirli  UM’  \S\  Ml  |irrri'nlilr  'i  iiii’s  should  ho  inli'r|iri'ii'd  wilh  caiilion. 
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